Building A Model - Search Criterion

Searching Random Model Space

Searching Random Model Space

       beta0      beta1      dist
1  0.8845133  1.6045973 22.814578
2 28.5444125 -4.9872329 37.977087
3  9.3374831  2.7945715  8.767417
4 36.8241342 -4.6211640 29.551357
5 -2.1537706  0.6587616 31.216438
6 36.3533108  0.3825870 10.693640

The Best Coefficients

Specifying a Formula

For R to perform these analyses for us, we need to be able to specify the function that we will use to represent the data (just like we did for plot(y~x)). Assuming the predictor is named x and the response is y, we have:

y ~ x

y ~ x1 + x2 

y ~ x1 + x2 + x1*x2

Fitting A Model

The easiest model.

Call:
lm(formula = y ~ x, data = df)

Coefficients:
(Intercept)            x  
     17.280        2.625  

Components within the Summary Object

Like cor.test(), the object returned from lm() and its summary object have several internal components that you may use.

 [1] "coefficients"  "residuals"     "effects"       "rank"         
 [5] "fitted.values" "assign"        "qr"            "df.residual"  
 [9] "xlevels"       "call"          "terms"         "model"        

 [1] "call"          "terms"         "residuals"     "coefficients" 
 [5] "aliased"       "sigma"         "df"            "r.squared"    
 [9] "adj.r.squared" "fstatistic"    "cov.unscaled" 

Components within the Summary Object

The probability can be found by looking at the data in the F-Statistic and then asking the F-distribution for the probability associated with the value of the test statistic and the degrees of freedom for both the model and the residuals.

   value    numdf    dendf 
16.56838  1.00000  8.00000 

The Summary Object

When you print out the summary( fit ) object, it uses the fstatistic to estimate a P-value. If we need that P-value directly, this is how it is done.

[1] 0.0035813

Model Diagnostics - Residuals

Normality Of the Data

Leverage

Decomposition of Variance

The terms in this table are:

• Degrees of Freedom (df): representing 1 degree of freedom for the model, and N-1 for the residuals.

Decomposition of Variance

The terms in this table are:

• Sums of Squared Deviations:
  • \(SS_{Total} = \sum_{i=1}^N (y_i - \bar{y})^2\)
  • \(SS_{Model} = \sum_{i=1}^N (\hat{y}_i - \bar{y})^2\), and
  • \(SS_{Residual} = SS_{Total} - SS_{Model}\)

Decomposition of Variance

The terms in this table are:

• Mean Squares (Standardization of the Sums of Squares for the degrees of freedom)
  • \(MS_{Model} = \frac{SS_{Model}}{df_{Model}}\)
  • \(MS_{Residual} = \frac{SS_{Residual}}{df_{Residual}}\)

Decomposition of Variance

The terms in this table are:

• The \(F\)-statistic is from a known distribution and is defined by the ratio of Mean Squared values.

• Pr(>F) is the probability associated the value of the \(F\)-statistic and is dependent upon the degrees of freedom for the model and residuals.

Decomposition of Variance

The classis ANOVA Table

Analysis of Variance Table

Response: y
          Df Sum Sq Mean Sq F value   Pr(>F)   
x          1 568.67  568.67  16.568 0.003581 **
Residuals  8 274.58   34.32                    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Relationship Between \(R^2\) & \(r\)

How much of the variation is explained?

\[ R^2 = \frac{SS_{Model}}{SS_{Total}} \]

     Regression R^2 Squared Correlation 
          0.6743782           0.6743782 

Helper Functions

Grabbing the predicted values \(\hat{y}\) from the model.

Helper Functions - Residuals

The residuals are the distances between the observed value and its corresponding value on the fitted line.

What Makes One Model Better

There are two parameters that we have already looked at that may help. These are:

• The \(P-value\): Models with smaller probabilities could be considered more informative.

• The \(R^2\): Models that explain more of the variation may be considered more informative.

New Data Set

Let’s start by looking at some air quality data, that is built into R as an example data set.

     Ozone           Solar.R           Wind             Temp      
 Min.   :  1.00   Min.   :  7.0   Min.   : 1.700   Min.   :56.00  
 1st Qu.: 18.00   1st Qu.:115.8   1st Qu.: 7.400   1st Qu.:72.00  
 Median : 31.50   Median :205.0   Median : 9.700   Median :79.00  
 Mean   : 42.13   Mean   :185.9   Mean   : 9.958   Mean   :77.88  
 3rd Qu.: 63.25   3rd Qu.:258.8   3rd Qu.:11.500   3rd Qu.:85.00  
 Max.   :168.00   Max.   :334.0   Max.   :20.700   Max.   :97.00  
 NA's   :37       NA's   :7                                       

Base Models - What Influences Ozone

Individually, we can estimate a set of first-order models.

\[ y = \beta_0 + \beta_1 x_1 + \epsilon \]

Base Models - What Influences Ozone

More Complicated Models

Multiple Regression Model - Including more than one predictors.

\(y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \epsilon\)

More Complicated Models

For Completeness

How about all the predictors. \(y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \epsilon\)

For Completeness

\(R^2\) Inflation

Any variable added to a model will be able to generate Sums of Squares (even if it is a small amount). So, adding variables may artifically inflate the Model Sums of Squares.

Example:

What happens if I add just random data to the regression models? How does \(R^2\) change?

Random Data Effects

Random Data Effects

Perfect - My Models RULE

I can just add random variables to my model and always get an awesome fit!

Not so fast Bevis!

Model Comparisons

Akaike Information Criterion (AIC) is a measurement that allows us to compare models while penalizing for adding new parameters.

\[AIC = -2 \ln L + 2p\]

The criterion here are to find models with the lowest (absolute) AIC values.

Model Comparisons

To compare, we evaluate the differences in AIC for alternative models.

\(\delta AIC = AIC - min( AIC )\)

Questions